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Introduction 


The unprecedented and unplanned shift to remote instruction in response to the 
COVID-19 pandemic in spring 2020 rocked higher education. For the first time, the 

great majority of postsecondary faculty and students engaged in teaching and learning 
entirely at a distance. Faculty members who had never taught (or learned through) online 
courses were challenged to teach in a new format. Students who had never taken online 
courses before, and were not expecting to do so, had to shift to learning at a distance at 
the same time they were coping with abrupt, pandemic-induced changes in their living 
circumstances. 


With support from the Bill & Melinda Gates Foundation, we set out to study the nature of teaching and 
learning in college courses during this period, with a focus on the experiences of students of color and 
those from low-income backgrounds. We reasoned that these students are more likely to have outdated 
computing devices and limited internet connectivity that would restrict their ability to participate online. 
In addition, students with health issues or disabilities and those who struggle with academic English, have 
weak math backgrounds, or lack confidence about their ability to learn, could be especially vulnerable to 
deficiencies in online course materials and to the loss of hands-on activities and face-to-face support. 


We also hypothesized that students in courses that had been using digital learning to a significant extent 
before the shift to remote instruction would find the transition to remote learning less unsettling. We 
took the opportunity to examine experiences in courses that had been using adaptive courseware as part 
of teaching and learning improvement efforts funded by the Bill & Melinda Gates Foundation. (Adaptive 
learning courseware, such as ALEKS, LearnSmart, and MyMathLab, organizes course content by learning 
objective and uses assessments of student performance on an ongoing basis to determine an individu- 
alized path through the course content. Students use courseware independently at times of their own 
choosing.) 


The abrupt transition to remote instruction in response to COVID-19 posed significant challenges for 
both students and instructors. This report provides data on the prevalence of the different kinds of chal- 
lenges students faced during the shift to remote instruction and the nature of spring 2020 courses from 
the perspectives of both students and instructors. These descriptions are complemented by survey data 
on the prevalence of online instructional practices that are generally recommended in the online learning 
literature and analyses of the relationship of these practices to student satisfaction with their course. 


As the pandemic has persisted, restrictions on classroom-based instruction have extended much longer 
than we envisioned in spring 2020. Colleges and universities are looking for ways to design courses 

that are resilient in the face of epidemics and other interruptions to campus-based teaching while also 
being equitable to all students. This report highlights challenges and equity issues that surfaced in spring 
2020 and that should be considered in course design and implementation, even beyond the pandemic. 
Practices associated with higher student satisfaction with their courses have applicability far beyond the 
first semester of remote learning during the pandemic. 


Digital Promise Lessons From Remote Learning During COVID-19 | 1 


This report describes findings from two research activities conducted concurrently: 


¢ Survey of a nationally representative sample of over 1,000 undergraduates who were taking online 
courses that included in-person meetings when they began and had to switch to entirely remote 
instruction. 


¢ Qualitative descriptions of 29 courses offered by 10 institutions, based on interviews and focus 
groups with students and instructors. 


We have presented an extensive set of data from the student survey in a previous report.* In this report 
we highlight selected survey findings and illustrate them with related descriptions of remote teaching 
practices and learning experiences from the course case studies. The survey data document the preva- 
lence of various challenges and instructional practices; the comments of students and faculty from case 
study courses help us appreciate the implications for specific courses. 


Taken together, the survey and case studies provide complementary findings with respect to four re- 
search questions: 


e« What challenges did students experience in learning online and how did those challenges vary for 
different kinds of learners? 


e¢ What was the nature of online learning in courses during the COVID-19 pandemic? 


¢ What instructional practices were associated with greater student satisfaction with their courses 
during the COVID-19 pandemic? 


¢ How did prior use of digital courseware in blended courses using adaptive courseware before 
COVID-19 affect the transition to remote instruction? 


‘ https://www.everylearnereverywhere.org/resources/suddenly-online-national-undergraduate-survey/ 
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Methodology 
Student Survey 
Sample 


The Survey of Student Perceptions of Remote Teaching and Learning, developed by Digital Promise in 
collaboration with Langer Research Associates, was administered to a national sample of U.S. under- 
graduates, age 18 and older, who were taking college courses for credit that included in-person class 
sessions when the COVID-19 pandemic hit and had to finish the course by learning at a distance. Ipsos 
Public Affairs conducted sampling and data collection for the survey via its online, probability- based 
KnowledgePanel®, which administers internet-based surveys to randomly recruited participants in U.S. 
households. 


Of the 1,709 panelists initially responding, 1,014 were qualified undergraduates taking for-credit college 
courses that started with in-person classes and later went fully online in spring 2020. Quality control 
flagged respondents who completed the survey in less than one-third of the median completion time of 
10 minutes, leading to the removal of six cases, for a final sample size of 1,008. Among the 1,008 panel- 
ists in the final survey sample, 62% reported on courses in STEM subjects. The weighted student survey 
sample was 23% Hispanic and 15% Black and had 37% of respondents from homes with annual incomes 
under $50,000. 


Method 


After answering screening questions designed to ascertain eligibility for the survey, respondents were 
asked to think about one particular course as they responded to the remaining survey questions. Because 
STEM (science, technology, engineering, and mathematics) courses typically have larger equity gaps in 
course outcomes (Blatt et al., 2020), the survey asked respondents to answer the survey questions for a 
STEM course if they were taking one. Students who were taking more than one STEM course were asked 
to respond for the one they considered most important for their future. Students who were not taking a 
STEM course were asked to respond for the non-STEM course they considered most important for their 
future. 


Field work was conducted between May 13 and June 1, 2020. The margin of error for the survey is 3.6 
percentage points for the full sample. Additional detail concerning the survey methodology appears in 
the appendix of this report. 


Course Case Studies 
Sample 


A convenience sample of 10 broad-access colleges and universities that had been working to redesign 
their gateway courses to include adaptive learning courseware was recruited for the qualitative com- 
ponent of this research. Each of these institutions identified one or more courses that had undergone 
redesign to include use of adaptive courseware. Institutional project leads invited instructors of course 
sections using adaptive courseware and any instructors of the same courses that had not been using 
adaptive courseware to be part of the COVID-19 teaching and learning research. Willing instructors were 
asked to send an email informing their students of the opportunity to participate in a focus group or 
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interview between April and June 2020. The email invited students to indicate their interest by filling out a 
web form maintained by Digital Promise. Digital Promise researchers then contacted students directly to 
schedule participation. Instructors were not informed of which of their students had agreed to participate 
in the focus group. There were no criteria for participation other than having been enrolled in a course 
that implemented adaptive courseware in at least one course section that semester. A limitation of this 
recruitment approach is that students who volunteered were not necessarily representative of all students 
in these courses: Those students most severely impacted by COVID-19 and most disconnected from 
their courses may not have even seen the invitation to participate. 


Ninety-two students taking 22 different courses at the 10 postsecondary institutions participated in an 
interview or focus group. Of these, 73 students were enrolled in a 4-year institution and 19 students were 
enrolled in a 2-year institution. Participating students were not asked to identify their race, ethnicity, or 
income level but likely reflected undergraduate enrollments at their institutions, which were 49% Black, 
Latinx, or Indigenous. Among full-time undergraduates at these institutions, 34% were on Pell grants for 
low-income students (note that non-U.S. citizens are not eligible for Pell grants). Most of these students 
(88%) were in sections of a course that had been using adaptive learning software prior to the COVID-19 
outbreak. In most of these case study courses, students’ work with the courseware counted for 30% or 
more of their course learning time, which meant that the course satisfied customary criteria for being 
considered a blended learning course (Allen et al., 2007). The case study courses came from the fields of 
biology, chemistry, economics, geology, mathematics, and statistics. 


In addition, 29 instructors from the same courses and institutions participated in interviews about how 
they conducted their course before and after the shift to completely remote instruction. The great 
majority of these (90%) had been teaching with adaptive courseware before COVID-19 forced campus 


closures. 
Method 


Depending on students’ schedules, they were either interviewed individually or participated in a focus 
group with up to four other students. In both interviews and focus groups, students were asked to 
describe different aspects of their course experiences pre- and during COVID-19. Interview questions 
asked these students how they used the courseware (if applicable) and how much time they spent using 
it before COVID-19 as well as afterward. Students in course sections that were not using adaptive learning 
software were asked about any digital learning tools that were used in their course. Students were asked 
about the computing device they typically used when working on the course and any technology issues 
they may have experienced. 


Questions about the course during COVID-19 included probes for how they accessed the internet, any 
changes to the computing device they used for the course, and any technology issues they experienced 
after instruction shifted online. We asked students how the structure and organization of their course 
changed once it was fully online (e.g., expectations for course participation, changes to course assign- 
ments or grading), for their perceptions of course difficulty, and what they missed about the course 

as it was taught before COVID-19. Students were also asked about challenges they encountered with 
continuing their learning at a distance (such as finding a quiet place to work) and how prepared they felt 
they were to take subsequent courses in the same subject area. Finally, students were asked about their 
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overall satisfaction level with the course during COVID-19 and, in the case of students in sections using 
adaptive learning systems, if they felt the use of the courseware before the coronavirus outbreak helped 
them continue their learning in the course after it was fully online. 


In their interviews, instructors were asked about the role of the adaptive learning system or other dig- 

ital learning tools in their course, their instructional practices pre- and during COVID-19, the biggest 
challenges they and their students encountered when transitioning to remote learning, and institutional 
supports the instructor received for transitioning the course. Instructors were also asked to reflect on 
how well their students had learned in the course and how well prepared their spring 2020 students were 
for future courses. 


Findings 
Compounding Equity Challenges When Learning Remotely 
Technology Access 


When courses moved unexpectedly to remote instruction, students no longer had access to internet 
connectivity and devices provided on college campuses. The survey data provide an indication of how 
widespread technology access issues were during the first semester of the COVID-19 pandemic. Internet 
connectivity issues were serious enough to interfere with students’ ability to attend or participate in their 
courses at least occasionally for 44% of students, with 16% of students reporting that they experienced 
such problems often or very often. 


Almost a quarter of students taking courses (23%) report- 
ed experiencing hardware or software problems serious One in six students taking 
enough to impact their ability to attend or participate in a course spring 2020 had 


their course at least occasionally, with 8% reporting that Haleclaateye connectivity Elalelola 
they had such problems often or very often. Hardware 


hardware/ software problems 
often or very often. 


and software problems tended to be experienced by the 
same students who had difficulties with their internet 
connection. Considering the two kinds of technology 


issues together, one in six students taking a course spring 
2020 had internet connectivity and/or hardware/ software problems often or very often. 


Reflecting long-standing inequities, technology access issues were more prevalent among students of 
color and those from lower-income households. Survey responses revealed that students from minori- 
tized race/ethnicity groups (comprised predominantly of African American and Hispanic students) were 
more likely to experience internet connectivity problems often or very often (22%) compared to students 
from non-minoritized groups (12%). Students from lower-income households (annual incomes under 
$50,000) were almost twice as likely to experience internet connectivity problems often or very often 
(22%) compared to students from households earning $100,000 or more (12%). 


The distribution of major hardware and software problems across different student groups followed the 
same pattern as that for internet connectivity problems. Larger proportions of racially minoritized stu- 
dents (12% combined) experienced hardware or software problems often or very often compared to just 
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5% of non-racially minoritized students. Frequency of serious hardware and software issues varied also by 
household income with students from lower-income households more than twice as likely to experience 
hardware problems often or very often (12%) compared to students from higher-income households 
(>$100,000 annual; 5%). 


Student comments during interviews and focus groups give texture to these numbers. One student in a 
case study chemistry course explained, “I've definitely had some issues with internet connection. | live 
pretty close to the mountains, so | get like one bar of service at all times. | can’t even make phone calls 
from my house, so | barely get any service." 


Another student described having to work in a less-than-ideal place to get internet access. 


I've been staying above my garage in my house and so internet access for me only works 
in my bathroom. | have to do a lot of my work like studying and watching the videos in my 
bathroom, which is inconvenient. 


Challenges Beyond Access 


After answering questions about their experiences with technology after instruction moved online, 
students completing the survey were given a list of other potential challenges to continuing their learning 
remotely and asked to indicate the extent to which each was a problem for them. Figure 1 shows survey 
responses for the six challenges included in the survey as experienced by respondents from different 
race/ethnicity groups. 


Maintaining motivation and focus. As illustrated in the figure, staying motivated to do well in the course 
was the most prevalent challenge to students’ completing their courses at a distance. The proportion of 
students who reported maintaining motivation as a major problem (42%) on the survey was double that 
for the next most frequently experienced challenge (finding a quiet place to work on the course, a major 
problem for 20%). 


Student comments in focus groups and interviews provided descriptive detail concerning how the 
challenges listed on the survey were manifested for different individuals. According to one math student, 
the most difficult part of the remote learning experience was listening to the lectures. “] would usually just 
zone out,” the student said. Another student described a similar challenge in their chemistry course. 


Well, for me one of the struggles has been focusing since you're staying in your own home 
so you get distracted easily and often enough. Also, at times just really connecting with the 
course. Before in person, you could connect with the course more than not being there, so 
that was one of the struggles | had. 
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Figure 1. Experience of problems participating in their course at a distance by race/ethnicity 


42% 
Staying motivated to do well 31% 
in the course ~ 45% 
48% 


Finding a quiet place where you 
could do the course online ~ 


Fitting the course in with your 
home/family responsibilities ~ 


Not knowing where to get help 
with the course ~ 


Feeling too unwell, physically or 
emotionally, to participate ~ 


Fitting the course in with your work _ 
schedule, if you work for pay 


0 20 40 60 


Many students described the temptation to put off watching lectures when there were no scheduled 
class meetings. One biology student explained, “What | miss is just the whole schedule and routine thing 
of going to class, kind of like some mandatory thing you have to do.” Another biology student explained, 


With meeting face-to-face, you're holding yourself accountable because you had to go 

to class to know what was going on. But when everything went online, it was easy to pro- 
crastinate and put off watching the lectures. | know | personally was stuck with watching, | 
think | had seven or eight lectures that | had to watch over the weekend before the final just 
because | found myself not really wanting to wake up early and watch them. 


Finding a quiet place. After the shift to remote instruction, many students found themselves working in 
an environment that was not conducive to learning. Students reported being distracted by elements of 
their environment when working on their course, often because of other people in the home. “Finding a 
quiet place was the worst thing ever,” said a math student. “There's dogs here. | live with my boyfriend. 
It's chaotic here. Finding a quiet place was definitely one of the biggest struggles.” A chemistry student 
felt similarly, “| get really distracted very easily,” said the student. “Having online lectures while my parents 
are stomping around upstairs and my dog's barking, | can’t focus well at all.” Another chemistry student 
described being distracted by their younger sibling. 


Digital Promise 


Lessons From Remote Learning During COVID-19 | 7 


My room is right beside my little brother's playroom. He's always on his Xbox screaming, and 
of course, my room is not as soundproof as much as | wish it was. The noise—I'd be trying 

to take a test or studying for a test and all of a sudden, a little sibling walks into my room and 
just completely throws off my train of thought ... So, I’m usually downstairs outside on the 
porch trying to find a quiet place to study. But of course, it’s Mississippi and it’s starting to get 
quite warm, so that’s not going to work for much longer. 


Students were particularly distressed by lack of a quiet place when taking course exams. A chemistry 
instructor shared some of the situational challenges that students experienced when taking their exam at 


home. 


During the exam | had one student who emailed me and she’s like, “I'm really sorry, but 
you re going to see that | was talking during my exam.” She's like, “| had warned my brother 
that | was taking an exam and he was screaming in the background and | couldn't concen- 
trate, and finally, | just got fed up and | snapped.” | had another student who emailed before- 
hand and said, “Look, | told my roommate, | have an exam, but both of our computers are in 
the same room. He doesnt seem that concerned that | have an exam, so if there's somebody 
talking in the back, I'm sorry. | tried.” He was actually one that we had to live Proctor and my 
TA [teaching assistant] emailed me and she’s like, “Somebody screaming in the background.” 
His friend was playing an online game and was just screaming and there's nothing they could 
do. They had to work through it. | have so many more examples of things like that, where | 
think that was a huge struggle. 


Competing family responsibilities. Conflicts with family responsibilities were deemed a major problem 
by 1 in 6 survey respondents. Students described the conflict between doing coursework and the extra 
layer of responsibilities for children or other family members at home. 


It’s a totally different environment, going from leaving the college to now being back with my 
parents and stuff like that. There's a lot more like needs for me at home, helping take care of 
my sister and helping to do chores and my parents want it done right then and right there. 
When | was at school, | knew exactly how | could plan for myself when | wanted to do things 
and how | wanted to do things. 


| would get really distracted by my family ... | live with my grandparents and my great-grand- 
parents. | would always have to help them out. | always have to cook for them, take care of 
them. Even simple things, like turning on the TV so they can watch the news. I'm African... 
my African parents, | always have to obey them, | can't tell them, “No”, that is disrespectful. | 
always have to make sure | do what they want. 
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Not knowing where to get help with the 
| have a two-year-old and we share the room, 


so when she takes her naps, that’s the only time 
| can really study. Right now, she’s next to me, 
I'm waiting for her to wake up. When | was on 


course. When courses switched to remote 
instruction, many students had to use 
learning and communication technologies 
that were new to them and encountered 


campus she was at daycare and | had a chance 
to just sit and focus just on school instead of me 
worrying if she’s okay. 


changes in course requirements and 
deadlines at the same time they lost 
in-person contact with instructors, peers, 
and teaching assistants. Although many 


colleges continued supplemental in- 

struction (SI) and tutoring services online, 

students did not necessarily know how to access those resources from a distance. Almost 1 in 6 survey 
respondents cited not knowing where to get help with the course as a major problem. Students in case 
study courses described their feelings of needing help they couldn't access. 


It was just really frustrating, | guess, to just [try to] understand stuff and not really know how 
to get the help you needed when you aren't seeing your professor or seeing your classmates 
or anything like that. 


Feeling too unwell. Roughly 1 in 7 survey respondents rated feeling unwell, physically or mentally, as a 
major problem when they were trying to complete their course online. In the words of several case study 
students, 


Probably, dealing with the different quarantine things, different regulations that they put in 
place: the six feet, the masks, and all of that. That really put me into a trouble because I’m not 
a good change person. | don't deal with change very well. It was all sudden, and they told us, 
“Stay inside.” 


With everything that’s going on, and going to the grocery store is dangerous now. It takes 
so much out of my day just to go run errands or something like that. If | have to leave the 
house, | have to wear protective gear and stuff like that. | have so much going on besides 
class ... the main thing for me right now is survival and staying healthy, but then, | have like 
all this difficult work that makes it more stressful for me, and just the environment and life is 
different, and school is still hard, if not harder. 


Conflicts with work. The proportion of survey respondents who cited conflict with work schedules as a 
major challenge to completing their course was relatively small (8%), but the impact on those particular 
students could be large. As a case study student in a chemistry class described, 


My schoolwork, | usually end up doing late nights or over nights on my days Off... !m at work 
from 8:00 a.m. to 7:00 p.m., four days a week. That takes away a good chunk of classes that 
! could probably take and then | have a child, so that takes away a good amount of time. | 
just really do make sure | calendar in when everything is due and then plan from there about 
what time block after | get off of work or overnight or on my days off that | can set aside for 
studying, for each class and getting work completed on time. 
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As described in Means and Neisler (2020b), the non-technology challenges posed by COVID-19 were 
experienced most acutely by students of color, females, and students from low-income backgrounds. 
Hispanic students reported higher frequencies of experiencing six out of the seven challenges covered 
in the survey, including fitting coursework into family and work responsibilities, finding a quiet place to 
work, knowing where to go for help, and feeling too 


z ; unwell physically or emotionally to focus on course- 
Hispanic students reported 


higher frequencies of with internet access, together with students from 
experiencing six out of the families with annual incomes below $50,000. Recent 


seven challenges covered in reports showing inequities in cases, hospitalizations, 
the survey. and fatalities due to COVID-19 help to explain why 


work. They were also most likely to have problems 


minoritized students, especially African American and 


Hispanic students, were most likely to cite their physical 
and mental health as major challenges to participation in coursework.’ As found in previous research 
(Wladis et al., 2018), female students, especially women of color, were more likely than others to have 
experienced disruption due to loss of childcare. 


Instructor Responses to the Challenges of Remote Teaching and Learning 


Instructors had very little time—sometimes just a day or two—to prepare for shifting their course online 

in the spring of 2020. Particularly those instructors who had never taught online before found themselves 
“making it up as they went along.” The course case studies provide examples of the kinds of changes and 
accommodations instructors made. 


Instructors in the interview sample were very aware of the kinds of technology issues faced by some of 
their students. A statistics instructor, for example, shared the experience of a student who was disadvan- 
taged because of limited internet access. 


| had a student who was doing fine before COVID. After COVID, | don’t remember exactly 
how many assignments he missed but | emailed and said, “Hey, | noticed that you haven't 
completed this assignment. Is there anything | can do?" He said, “Well, we're struggling 

with the internet here. | don’t know what to do to complete them." | said, “| can extend the 
deadlines and you can even drive to the school parking lot. If you have a laptop, then you 
can use the campus WiFi, or even use the WiFi at a McDonald's inside your car or something.” 
Again, these are things that if a student doesnt have a laptop, then they can't do that. 


Instructor awareness of their students’ technology issues influenced the way they conducted their 
courses. In addition to extending deadlines (as the statistics professor above did), instructors often made 
their course materials available asynchronously, citing students’ technology access issues as the rationale. 


? https://www.cdc.gov/coronavirus/2019-ncov/covid-data/investigations-discovery/hospitalization-death-by-race-ethnicity.html 
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| realized that, of course, not all students have the hardware for Zoom. Phones are pretty 
ubiquitous and even computers are fairly ubiquitous, but WiFi is not. A lot of students really 
rely on public spaces in the university for WiFi. WIFI is not ubiquitous. Having a working mic 
and camera is not ubiquitous, or even speakers, is not necessarily ubiquitous. Understanding 
that, | would offer the classes via Zoom, but | would also post the recordings online. That 
way, if a student couldnt access a live Zoom recording because their hours at work changed, 
or they don't have access to WiFi, or whatever it may be, they could at some point stream 
that later. There was this synchronous lecture, but it could also be accessed asynchronously. 


Content Delivery 


Out of concern for students with limited technology access and those in different time zones or com- 
peting responsibilities, a number of higher education institutions (including two of the ten in our case 
studies) either recommended or mandated that all remote instruction for courses that had previously held 
in-class meetings be done asynchronously. A chemistry instructor echoed this concern that live lectures 
would not be equitable for all students in their interview, “I did want to do synchronous classes, but then 

| recognized that not all students would be able to participate, and | didn't want to put the students that 
said they weren't able to attend in real time at a disadvantage.” In the end, this instructor did hold live 
synchronous lectures but recorded them for use by students who could not attend during the regular 
scheduled lecture time. 


Building in options by offering lectures both in real time and as recordings posted online emerged as a 
common strategy at the case study institutions. As a biology instructor described, 


The first thing | did [after COVID-19] was record my lectures and deliver them asynchronous- 
ly, and that was per our department mandate. They requested that we do that. After hearing 
from some of the students who might've struggled with the technology, | do think that at 
least recording them was a good idea, but | also heard from some students who said it was 
really hard for them to switch to online. 


The national survey data suggested that this strategy of using both synchronous and asynchronous 
delivery within a course was common. The percentages of student survey respondents reporting use of 
synchronous and asynchronous delivery mechanisms in their courses shown in Table 1 would be mathe- 
matically impossible if many courses did not include both. 


Digital Promise Lessons From Remote Learning During COVID-19 | 11 


Table 1. Delivery mechanisms in courses taught remotely during COVID-19 


Delivery Element Percentage of 
Students Reporting 


(n = 1,008) 


Live sessions (synchronous instruction) 


Recorded lectures by the instructor (asynchronous) 
Pre-recorded videos, not of instructor 


However, a number of instructors and students in the case study courses expressed reservations about 


asynchronous instruction. A chemistry instructor, for example, reflected, “I feel like having the account- 
ability of being there [lecture] at the time, kind of getting it live, is definitely different than just watching 
the video secondhand.” According to a geology instructor, synchronous lectures were preferable because 
students would be less likely to procrastinate. 


| dont like asynchronous courses. It could just be a bias, but just like a goldfish will fill up to 
whatever size tank you put it in, a student will procrastinate to whatever deadline you give 
them. The students think, “If it’s asynchronous, well, then | can do it later.” We have to be as 
synchronous as possible. We still need to be asynchronous, but also offer a realistic avenue 
for those who can do it synchronously. 


Some students’ experiences resonated with this instructor's concern about the potential for procrastina- 
tion. As one biology student described, 


With meeting face-to-face, you're holding yourself accountable because you had to go 

to class to know what was going on. But when everything went online, it was easy to pro- 
crastinate and put off watching the lectures. | know | personally was stuck with watching, | 
think | had seven or eight lectures that | had to watch over the weekend before the final just 
because | found myself not really wanting to wake-up early and watch them. 


The biggest challenge after COVID was just keeping up because when it was in-person, we 
would meet twice a week. But once the videos were posted online, then we could access 
them whenever we wanted, so it was really easy to push it off and say, “Ill do that the next 
day,” and that was just kind of a cycle. 


Like the students responding to the national survey, many students in the case study courses described 
their motivation and focus as suffering during COVID-19 without “the reinforcement of having to show 
up to class.” In the words of one student, 
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It was definitely a little bit hard to maintain the same level of focus [after COVID-19]. | was 
able to get through the lectures and | took notes and everything still, but at times it was 
pretty painful to do. It really took a lot of motivation to really sit through the full extent 
without fighting my mind wandering or whatever, and | didnt have that at all when we were 
in the course. | was fully engaged in it while we were in person. 


Student and instructor reflections on their experiences with courses during COVID-19 suggest that there 
is a trade-off between providing students with maximum flexibility to do their coursework whenever 
and wherever they could and having a predictable schedule to keep students involved in the course. 


Having course lectures available both in real time and as recordings did not completely resolve this 
tension between equity and motivation. Some students who had access to both synchronous and asyn- 
chronous options for viewing lectures described the latter as impairing their motivation. A biology student 
explained, “What | miss is just the whole schedule and routine thing of going to class, kind of like some 
mandatory thing you have to do.” Another student described a similar challenge in their chemistry course, 


Well, for me one of the struggles has been focusing since you re staying in your own home 
so you get distracted easily and often enough. Also, at times just really connecting with the 
course. Before in person, you could connect with the course more than not being there, so 
that was one of the struggles | had. 


Other case study course faculty chose to create videos that resembled the in-class experience, in some 
cases showing themselves lecturing at a podium in an empty classroom or recording a set of PowerPoint 
slides with an inset of their face as they gave the lecture associated with the slides. While this approach 
provided an element of continuity with the course experience pre-COVID, it did mean that students were 
viewing long lectures online. 


A number of students in interviews and focus groups commented on the length of recorded course 
lecture material as an impediment to maintaining focus on course content. According to one math 
student, the most difficult 


Being at home made coursework really uninteresting Par Oriente te atid) 


because | could only listen to a lecture for 20 minutes at a 
time, and then when | went back to it, | wasn’t remembering the lectures. “I would usually 
what | heard from the last 20 minutes and all that. just zone out,” the student 
said. Similarly, a biology 


experience was listening to 


student taking a course where 


the instructor recorded longer 
video lectures described 
feeling less engaged with 

the material. 
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Laboratory Experiences at a Distance 


Hands-on laboratory experiences are an important part of many STEM and health sciences courses. 
Students expressed concern that remote instruction meant missing out on important lab work. One 
approach to dealing with the lab aspect of STEM courses was to have students view videos of the lab 
exercises being done by others, sometimes with instructor notations or requirements to answer some 


questions. 


| think my concern [with the course post-COVID-19] lies mostly with the labs because 
obviously labs are entirely hands-on and then when that was no longer an option. We're just 
watching people do all of these procedures, but more than likely, we're going to have to 
know how to do [them] in future biology programs. We didn’t get to fill gel wells and do elec- 
tro-freezes or dissect anything, which all seems pretty important for future classes, especially 
because you're just going to go further into those issues. 


For labs, [what] he [instructor] did was he would find someone doing the lab on a YouTube 
video or something and he would post it, and we would watch it. He would give us important 
details like mass of a liter or something, and we had to find stuff like the volumes and stuff 


within it. 


| was doing research in microbiology with an assigned microbe for use in in vitro testing with 
wax worms. Due to leaving campus, | had to throw my bacterial cultures in the trash and 
“did” the lab virtually using a PowerPoint our professor put voice recordings over. We still had 
to write a research paper, but our professor had to make the rules more lax because we had 
no actual time in lab. 


Oh, my bio lab was not very good, | guess. | didn't really like it in general. It’s said to be one 
of the harder labs for biology. We just watched this presentation video and filled out some 
questions on a document and submitted it every week. | didnt really feel like that was very 
effective learning. 


Online lab exercises appeared to be more effective in students’ eyes when instructors were present 
online and used scaffolds and probes to direct attention to important aspects of the lab procedure. As 
one biology student explained, 


The instructions are still there. They ask questions throughout it that you can. ... There's 
questions that they ask you on the online lab before you start the lab so that you understand 
the material better. Then, during the lab, they ask you questions to make sure if you're putting 
water in this, why are you putting water in this. If you're putting a cotton ball in between this 
acid and this, what are you doing, why are you doing it, what's the point. You understand the 
concepts better and how it relates to what you're doing. Then, they ask you questions at the 
end to see if you actually learned something. The format was the same and they also had 
ways of connecting the material that made it really easy for me to continue using it without 
struggling. 
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Other instructors were able to have students do some lab work at home using common household items 
or online lab simulations. An anatomy instructor explained, 


A couple of the GAs [graduate assistants] reported that the reflexology [lab] still went pretty 
well. They just asked them [students] to get a ruler from home, grab a partner, “Here this can 
work." Instead of having the patellar hammer, the tap on the knee. You can get your own little 
hammer, just something to knock the front of the knee, and so, a couple of them reported 
that they [students] still enjoy doing the experiment at home. 


Several instructors reported having students work with virtual laboratories available in the course learning 
software. One student remarked, “My chem lab was an actual simulation and that was cool.” 


Learning with Peers 


When students in case study courses were asked what they missed about their pre-COVID-19 course, 
many pointed to collaborating with peers. “I miss being able to talk to other people about the course,” 
said a biology student. “If | didn’t understand something, | could turn to my friend sitting next to me and 
be like, ‘Oh, did you get that, do you 
understand that?” Another student | miss seeing my fellow peers. Working the problems 
felt similarly about their mathematics out with the two other students that | was with helped 
course: "| miss asking questions and me learn better and faster. ...There’s someone else 


there instead of me just teaching myself or doing it 
[coursework] myself. 


being able to ask my classmates if 
they understand something,” said 


the student. “If maybe | don't under- 
stand the teacher's way of teaching, 
maybe a student's way is easier for 
me to grasp the concept." 


Another mathematics student also valued peer support and noted that it was more difficult at a distance 
even for study groups before the campus shut down. 


| definitely miss the atmosphere of the classroom ... The very first day of school, they lin- 
structor] told us to go around the classroom and figure out who we're going to work with, 
and we made a group chat. We were very involved as classmates | would say. | definitely miss 
that because if | had a problem or if | didn’t understand something, a friend was right next to 
me and they'd be able to help me. It was definitely more difficult to get in contact with those 
people once everything went down. 


In their interviews, course instructors were asked to describe the collaborative learning opportunities 
they provided for students, both before and after COVID-19. Few instructors had examples of including 
collaborative activities during remote learning, even when such activities were a part of the course before 
the coronavirus. For example, a biology instructor had a collaborative pre-COVID-19 iClicker activity 
where students worked in pairs or small groups to answer questions. After the coronavirus outbreak, 

the iClicker activity was replaced with online weekly quizzes for which students submitted their answers 
individually. A student in a chemistry course that similarly replaced a peer learning activity with embedded 
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quizzes after shifting to remote instruction explained the implications of losing opportunities to discuss 
course content with peers. 


When | was in class the professor would put one [question] up on the projector and be like, 
“Okay, answer this question.” | would be able to talk to people around me and even discuss 
further in detail why the other answers are wrong but now since it’s online, it kind of takes 
away the [discussion] element because after they [professor] present the question they say 
the answer and then we just submit our answers into a quiz in Canvas. It takes away the 
learning element of it, | guess. 


Instructors’ reservations about requiring students to work on the course at any set time, as described 
above, also led to a reduction in peer-to-peer interaction in class discussions. An economics professor 
who normally devoted the majority of class sessions to interactive discussion explained, “One of the not 
so good things about teaching during [COVID-19] is that | ended up having to abandon our class discus- 
sions because | couldn't require that everybody be there at the same time.” 


Student Support 


When courses switched to remote instruction, many students had to use learning and communication 
technologies that were new to them and encountered changes in course requirements and deadlines at 
the same time they lost in-person contact with instructors, peers, and teaching assistants. Although many 
colleges continued supplemental instruction (SI) and tutoring services online, students did not necessarily 
know how to access those resources from a distance. The survey data presented above showed that al- 
most 1 in 6 undergraduates nationally viewed not Knowing where to get help with their course as a major 
problem. Students in case study courses described their feelings of needing help they couldn't access. 


It was just really frustrating, | guess, to just [try to] understand stuff and not really know how 
to get the help you needed when you aren't seeing your professor or seeing your classmates 
or anything like that." 


| think the hardest part is not being able to get that extra help on the topic, like going to SI! 
[supplemental instruction] sessions or tutoring. | think that’s just the hardest part about online 
classes. You could go online during a certain hour and have a Zoom meeting, but | didn't 
think it was as useful as an actual face-to-face one. | tried that way and for tutoring, but the 
meetings were always filling up quickly, so it was hard; you had to compete to get attention. 


One student described feeling uncomfortable asking questions during Zoom meetings in their math class. 


Well, being in class gives you the opportunity to ask questions right there. With the Zoom 
experience, | felt like | was interrupting the whole class. It wasn't the same thing for her [the 
instructor] to answer your questions right then and there like when we're in class. 


Students missed being able to ask questions in real-time about their performance or their understanding 
of course topics. A math student explained, “I just miss being able to ask a question in that moment to 
elaborate on something.” An economics student shared, 
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| really missed the fact that if | had a question, right there and then | could put my hand up 
and ask a lecturer. Besides that, | miss the human-to-human interaction in general in the 
class setting because there's a certain level of spontaneity, | guess, that you would have. 
Sometimes a colleague will have a question that you wouldn't even think about at that time. 
They ll be able to say, “Hey, what about this?” And then you'd be like, “Yes, | didn’t think about 
that either. What about that?" 


Students pointed out that the lack of immediate feedback was particularly deleterious to laboratory 
activities. 


There was not a good way to get help while working on any of the lab portions. [It was] Hard 
to know if you were doing things correctly with proper outcomes. 


Instructors too found the loss of in-person connections difficult. Asked what the most challenging thing 
about switching to remote instruction during COVID-19 was, an economics instructor responded, 


Probably not seeing them [students] because in a blended class I'm expecting to see you 
once a week... It was like they were all scattered. | didn't see their faces. If | see you in person 
once a week, | could see if something's troubling you, | can see if you're struggling, | can see 
if you're stressed. I'm right there, so I'm reading body language as well as doing the teaching 
thing, but | didnt have that. 


Students often sent questions to their instructors through email. Their satisfaction with this method 
depended on the instructor's response time and response predictability. A biology lab student expressed 
frustration about the delayed response time getting answers to questions about lab assignments. 


There wasnt consistent communication all the time with the professor. There was just a lack 
of clarity of, “Okay, | have a question now, but when am ! going to get this response?” It could 
be anytime between one or two days and the lab is due today. If | have a question about the 
lab, why even bother asking a question because I’m not going to get a response on time. 


Instructors, on the other hand, often felt worn down by student expectations to be available to answer 


questions 24/7. 


Yes, | would say | found myself working a lot more post-COVID than prior to COVID. There 
were a couple of students that expected responses to emails or to questions in the discus- 
sion forums within 10 minutes of them posting it. It'd be 5:30 at night and we would try to 
eat dinner and Id be like, “| got to respond to this.” It was tough because there was no, “This 
is the end of the day work balance, and then okay, | can focus on my family.” That was no 
longer there. 
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Assessment 


Adapting assessment and grading practices to the circumstances of spring 2020 was another challenge 
for instructors. Most faculty were sensitive to the stressors on many of their students and wanted to avoid 
adding to those burdens through practices that would unfairly penalize students with limited technology 
access or competing responsibilities. On the other hand, instructors value academic rigor and integrity 
and wanted their assessment and grading practices to motivate their students’ continued engagement 
and learning. Balancing compassion with standards that motivate hard work was a continuing chal- 
lenge for instructors during COVID-19. 


Case study interviews revealed widespread adjustments to course assignments and due dates. A statistics 
instructor, for example, described the reasoning behind being more flexible with due dates during the 
pandemic. “A lot of my students have families and work and a lot of different responsibilities,” the instruc- 
tor said. “I found myself being even more flexible with regards to deadlines. If a student requested to have 
an extension, | would be like, ‘Yes, sure. Go for it. Definitely.” A mathematics instructor similarly described 
widespread relaxation of deadlines, “| would say I've had to reopen assignments that were not submitted 
by the deadline for two-thirds of the class at some point, and that was okay.” 


Students appreciated their instructors’ flexibility around grading and deadlines. “I did like how he [profes- 
sor] showed sympathy if we wanted to add an additional day for a test,” said a math student. "Whenever 
we would have a test, he would ask us if we wanted to open it for us for an extra day, and that was very 
helpful.” A biology student described how the deadlines for quizzes were extended. 


| definitely noticed that there were a lot of extensions allowed. Those quizzes that were 
supposed to be due on Fridays, | think my professor must have gotten bombarded by a lot 
of complaints about that deadline, because, | think, every single one of them got extended 
to a further date. | think she was trying to be flexible and willing to accommodate, which, I'm 
sure, a lot of students appreciated. 


Other students described being allowed to replace their lowest grade on an assignment or exam. In the 
words of a chemistry student, 


Originally, we had four exams and then one early exam that made up an enormous part of 
your grade. After the Coronavirus outbreak, she [professor] was going to drop our lowest 
exam score and just use our other three just to relieve some of that stress. 


Technology for proctoring tests given online was used widely during COVID-19, but not without con- 
troversy (Doffman, 2020). Some institutions and faculty objected to the costs of the proctoring service. 
Others noted that the technology access differences that affected students’ learning also come to the 
fore when exams are given at a distance. In one chemistry department, two instructors decided to use 
Respondus, a lockdown browser that controls students’ testing environment within a learning manage- 
ment system. One of the instructors explained that students needed to have a webcam so they could be 
recorded through the lockdown browser. Those students who didn't have a webcam were linked through 
Blackboard Collaborate to a teaching assistant who watched students on video through their phones as 
they took the exam. 
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Some students expressed anxiety or resentment about having to use a lockdown browser system for 


taking their exams. 


| think my greatest challenge was motivation and using the ProctorU system. We were forced 
to use this proctoring service and after reading the privacy policy | did not agree with what | 
was signing up for but was told I'd fail the tests if | didn't. It’s a really sketchy system and can 
see so much of your personal information on your screen | really think they could've found a 
much better alternative. | felt like my rights were taken away when | was forced to sign up for 
Proctorvu. 


An alternative to lockdown browsers used by some instructors was the imposition of relatively tight 
windows for completing the exam. A chemistry instructor explained the rationale behind this approach. 


Students could take the exam unlimited times, but each time the student was limited to an 
hour. | didn’t want them [students] to have it open for 24 hours and look everything up. They 
could have used their book, they could have Googled things, but | tried to use the time limit 
to squash how much they could look things up. 


Short time windows for completing course examinations had the downside of further contributing to 
inequity for students with substandard internet access. A chemistry student described the emotional toll 
of never Knowing when the internet connection might fail. 


With the unstable internet that | have, | go in with more anxiety than | would normally if | 
were in person because | don't know whether my internet is going to go out or I'll have to 
contact someone to get the test reset, and then see if they're going to let me retake my test. 


Rather than using short timeframes or lock-down browsers, some instructors decided to change to 
open-book examinations. A mathematics instructor, for example, switched to using an open-book format 
for the final exam. 


Once we couldn't do [proctoring] anymore, we then needed to switch to online exams. | had 
static exams, meaning everybody got the same questions. | just told students, “Hey, you can 
have your notes available. You can use your book,” because | didn't feel it was reasonable to 
expect that they're not going to do that. 


Another mathematics instructor who also decided to use an open-book format did so primarily out of a 
sense of fairness. 


If anything, my expectations for the exam were a little higher since | told them... “It’s an open 
book, open notes [exam]. Please don't use the internet.” I'm not going to tell them that [they 
can't use these resources]. If you tell the students they can't use their notes and they're sitting 
at home, the good ones that don't ... they're going to be penalized. Cheaters are going to 
cheat. | didn't feel comfortable with that. 
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Students had mixed reactions to changes in their exam format. While some expressed nervousness or 
resentment about exam proctoring systems, there were others with unproctored exams who felt there 
was no longer a good reason to study. In the words of one chemistry student, 


Now we're having to take tests and quizzes and stuff at home, and they're not being proc- 
tored or anything. It’s so easy to cheat or to go online, and you really don't have to study for 
any of your tests. Now, you're just not really learning much of anything. 


While making these changes in test timing and security measures, instructors reported few changes in the 
content of the assessment items and tasks they gave their students in spring 2020. 


Perceptions of Student Learning During COVID-19 


Instructors and the majority of students in case study courses were fairly sanguine about the adequacy 
of student learning in these courses. Only 3 of the 29 instructors we interviewed stated outright that 
they did not think students who took their course in spring 2020 would be prepared for the next course 
in the sequence. Just over half of the case study course instructors gave positive replies when asked how 
well their spring 2020 students were prepared for future coursework (other instructors said they were not 
sure). A statistics instructor, for example, commented “If anything | think they were a little better, maybe, 
learned a little better.” A biology instructor gave a positive response that was a bit more guarded, 


The material in 181 [biology] seems to be recurrent material in bits and pieces throughout the 
career of a biology major, of a student majoring in biology, so | think they will have a chance 
to refresh. There is, | think, a solid foundation in the sense that a student will be able to go 
back and revisit and say, “Oh, yes, | remember this,” but | am not sure if they will remember 
right away. 


Instructor perceptions of student learning during COVID-19 may reflect their encounters with students 
who remained active in the course, thus under-representing the experiences of students most vulnerable 
to the interruption of regular routines and supports. A number of instructors did stipulate that their 
optimism that students had learned the course material remotely was confined to those students who 
continued participating in course activities during COVID-19. 


| try to structure the activities so that you can't help but learn something when you're doing 
this and then that builds that encouragement and helps them. | think that there were a lot of 
tools and the ones that came prepared to learn and did the [assigned activities]—they learned 
something. 


Almost three-quarters of the students in interviews and 
focus groups said they thought they had still learned Almost three-quarters (o) i 
the course material despite having to finish their course the students in interviews 


remotely. This compares to 57% of students in the Elalom refers is fe] ge)¥] ok Tel they 
national survey sample who were somewhat or very 


thought they had still learned 
the course material. 


satisfied with their learning overall during COVID-19. 
The positivity of students in the case study courses 
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should be interpreted in light of the fact that they volunteered to be interviewed; other students in these 
courses may have been more pessimistic. Moreover, more than a quarter of the students in focus groups 
and interviews expressed concern that they had not learned as much as they would have if the course 
had continued with in-person classes. Students in STEM courses who were majoring in that subject 
were particularly likely to voice concerns about how well they would fare in future courses in their 
major. 


This [remote learning] is kind of what's been making me reconsider being a biology student. 
It's just because learning online is so difficult. This is my 103. It’s a 100-level biology class and 
everything that | learn after it is going to be based on the foundations that | learn in this class. 
I'm not learning these foundations very well, so it’s like it’s going to make everything after this 
harder because | didnt learn any of this the way that | was supposed to or as well as | should 
have. 


I'm worried. My major is chemical engineering and | feel like this is probably the most 
important class to really have a mastery of because it is the foundation for everything, every 
other class I'm taking in the future. So, I’m definitely concerned that next year is going to be 
a struggle because I'm taking a whole bunch of kinds of chemistry classes and without fully 
understanding what I'm supposed to be learning right now, | definitely think it’s going to be a 
big setback. 


I'm going to Organic Chem in the fall. Not having the basics, it’s going to hurt. I’m going 
to have to devote way more time to chemistry now than I’m pretty sure the typical per- 
son would have to because I'm going to have all these little soaces missing at the end of 
[Chemistry] 105, which is basically the foundation for the rest. 


Mitigating the Challenges — Instructors’ Use of Recommended Practices 
Adjusting Content Delivery to Online Instruction 


In student focus groups and interviews, we found that students taking courses with synchronous live 
class sessions did not report the same level of difficulty with maintaining motivation or focus as 

those with entirely asynchronous courses did. Fewer than 5% of case study students with synchronous 
sessions brought up difficulties keeping up with the course compared to close to 25% of those in totally 
asynchronous courses. For the former, having their online lectures scheduled at a regular time each 
week contributed to their overall satisfaction with the course. For courses with asynchronous elements, 
students expressed a preference for short instructional segments. A number of students in interviews and 
focus groups cited the length of recorded course lectures as an impediment to maintaining focus on 
course content. 


Students’ preference for short segments when learning asynchronously is congruent with instructional 
design guidelines for online courses (Scagnoli, 2012). Some of the instructors of case study courses 
followed this advice in spring 2020 when they moved their courses to remote delivery. One geology 
instructor, for example, broke each course lecture up into two to three shorter videos that were 10-25 
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minutes long. He inserted blank screens 
Being at home made coursework really uninteresting within each video containing digital 


because | could only listen to a lecture for 20 minutes notes oa whatwould have heanwiiten 


at a time, and then when | went back to it, | wasn't ei taecelace blacieaardl. & miaihomatics 


remembering what | heard from the last 20 minutes 
and all that. 


instructor using an asynchronous course 
format went a step farther, creating 


short instructional videos with answers 
for some of the questions that students 
had about the statistics software they 
were using. 


A student in a biology class was appreciative that the instructor did not record full-length lectures but 
instead created a series of shorter videos: 


! thought [the professor] made a good choice for this class in breaking up the lectures. In the 
classroom setting when we had 50 minutes, we might've got between 30 and 40 minutes of 
actual lecture time. But rather than post a 35- or 40-minute lecture, they posted the same 
amount of lecture material but they broke them up into, like, eight videos. Each one was 
more of a bite-size ... | have a hearing disability. In class | would've been one of the people 
interrupting [the professor] to make them repeat something. Once we were fully online, they 
wouldn't have been able to get through the same amount of lecture material in the same 
amount of time. Having one long lecture wouldn't have worked ... so, | appreciated that 
choice that they made with the lectures. 


Admittedly, creating new course materials in smaller chunks placed a burden on instructors at a time 
when they too were impacted by the COVID-19 pandemic. As one instructor explained, 


| found myself creating little video explanations for some of their [students] questions ... 
Going through those steps while you're watching the person was much more helpful than 
just being like, “Go here and click on there. | went through the whole thing step by step ... 
It required a lot of work outside of class, on my end, to make sure that learning was still 
occurring. 


Instructors who used adaptive courseware before COVID-19 described fewer challenges than their 
colleagues did in preparing their courses for the transition to remote instruction. As a math instructor 
explained, 


Because we're already so web-based, as far as our content goes, it wasn‘ that long [to 
prepare the course to go online]. It was pretty easy to switch it over. | already had all their 
notes online. | already had Collaborate [the videoconferencing function in the Blackboard 
learning management system] set up. | didn't have to do anything with my grading for that 
particular semester. | just had to put together a few documents and send out a few emails. It 
was really relatively easy for me. It was just, “Let’s roll" ... | would say that | spent maybe a few 
days, maybe eight hours. 


Digital Promise 
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These instructors also reported that the transition to remote learning was fairly easy for their students. “In 
some ways, | think that we were lucky that we had a lot of online stuff set up already,” said a chemistry in- 
structor. “As our schools transitioned, obviously, the lecture portion wasn't happening anymore, but a lot 
of their homework remained the same, so it was just substituting in those online lectures.” A geography 
instructor shared a similar comment about their course: “The students have used that [courseware] as 
their ‘textbook’ for the last two academic years. They were already fully doing all of their reading materials 
and their interactivity online before all this happened.” 


The mitigating role of prior use of adaptive courseware surfaced also in the interviews and focus groups 
with students. “I think for the most part they just kept it [course] consistent in terms of how they taught 

the class,” a biology student said. “They [professor] were able to just keep things going smoothly.” For a 

chemistry student the prior use of adaptive courseware similarly eased the transition to remote learning 
and increased their course satisfaction: 


This class is one of my classes that transitioned very well and has done a good job of pro- 
viding lectures at the same time period ... it was probably one of the smoother transitions 
because our homework was already online. | would say I'm very satisfied with this course. 


Including Active Learning Activities 


The incorporation of active learning in a college course is one of the best-researched strategies for 
improving student learning (Freeman et al., 2014; Haak et al., 2011). The sudden shift to remote instruc- 
tion in spring 2020 was not conducive to major changes in course design to make learning online more 
active, but many of the instructors already using active learning practices in their classroom-based 
courses did seek ways to preserve those elements after their courses went online. 


To support students in maintaining motivation and focus, a number of instructors embedded a guided or 
scaffolded note-taking activity or quizzes that students were to complete while watching lecture videos. 
Completing these notes or quizzes often counted toward the student's grade and had to be completed 
on a specified schedule. A biology instructor described the reasoning behind this practice. 


There was a lot of active learning, so they [students] would watch videos and turn in notes ... 
students love watching videos but they never extract the information from them. They think 
it just permeates to them, and so | just literally took the script and ask questions about the 
script so that they would extract the information that the video was trying to give them, and 
that was it. It was a very simple assignment but it [learning] went from passive to active. 


A geology instructor assigned a for-credit note-taking activity in conjunction with their short video 
lectures after the course went online. The instructor used this assignment to track students’ engagement 
with the course. 


The students were required to fill out a guided lecture outline that takes them through the 
notes in the presentation with me and they submit that on Bb [Blackboard] Learn as an 
assignment for their attendance ... it’s basically an outline where the student can just follow 
along and fill in the blanks so they know how to organize their notes, they know what's 
important to write down, they know what's not important. 
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Focus group students in courses incorporating guided note-taking or other active learning activities 
during lectures did not report the same level of difficulty with motivation or staying engaged that 
other students did. |n most cases, required note-taking activities were a continuation of activities used in 
the course before COVID-19, which helped students maintain a routine of working steadily on the course 
after the transition to remote learning. As a biology student explained, 


Because of the way the class was structured already, it was pretty easy for all of that to stay 
the same because there were still days when you were supposed to post your notes. That 
meant you had to listen to all those video lectures and take notes on them, and then scan 
them and post them online by the due date. That kept that expectation pretty consistent. 


For another biology student, the note-taking activity was one of several assignments that were assigned 
with each weekly lecture. The student described the sum of these activities as being “task heavy” but 
acknowledged that they helped with learning the course material. 


In any given week you had, like, a video assignment that you had to review, take notes on, 
scan the notes, and then answer questions on it. Then, you'd have a concept map which 
was— again, it was a document that you had to fill out with the concepts of that week, and 
then scan it and post that. ... You'd have to print notes to be ready for the lectures, because 
you had to take your written notes from the lectures on these specific pages. Then, you‘d 
have to scan those and post them. You'd have a vocabulary quiz. Then, you'd have a weekly 
quiz as well. ... We did all of those different types of assignments every week. There was never 
a week that you didn't have that many of those. They hit every single one of those different 
types of learning outcomes every single week. Like | said, it ended up being somewhat task- 
heavy but at the same time, because you were reviewing all the materials in all those different 
ways, you couldnt help but learn it. 


Prompts for reflecting on their own learning is another strategy to encourage active learning. An 
example of how reflection activities can be built into a course was provided by the instructor of an 
economics course. 


| do have an extra credit feedback survey at the end of each of my classes. ... Then the next 
question is, tell me one thing you will remember a year from now from this class, which 
translates to something | really understood. Then tell me one thing you still don’t understand. 
| read all of those at the end of each class and if there's a trend [like] they really didn’t under- 
stand the elasticity the way | thought they did, then | try to augment or change or look at, 
well, how am | going to teach the elasticity next time? 


A mathematics instructor was one of the few faculty who described successfully transferring a col- 
laborative learning experience into the online environment. Before COVID-19, the course included a 
recitation session that began immediately after lecture and was attended by the same students. Students 
formed small groups to work on assignments together during these sessions and if they had questions, 
they could ask the instructor or an undergraduate learning assistant, both of whom circulated the room. 
During COVID-19, the instructor continued the activity through a live synchronous session that made use 
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of Zoom’'s "breakout" groups. During breakout sessions, students worked on problems in small groups 
for approximately 12 minutes, after which they came back together as a larger group in Zoom to discuss 
the problem as a class. Students were then switched to a different breakout group to work on the next 
problem. 


More extended collaborative learning occurs when students engage in project-based learning, meeting 
outside of class time to work on a problem solution or joint product. On the national survey, 33% of stu- 
dents reported that their course instructor had them work on group projects outside of scheduled class 
time during COVID-19. Among the 29 instructors who were interviewed about the case study courses, 
however, none described a course assignment or activity where students worked collaboratively outside 
of scheduled class or recitation time to produce a joint product or solution when the course was fully 
online. (Students responding to the survey may have interpreted the word “project” quite broadly.) 


Supporting Students Online 


Many instructors sought to mitigate loss of opportunities to ask questions in class by holding online office 
hours or review sessions with their students. A chemistry instructor, for example, described using Zoom 
for office hours and recitation help sessions. 


| had Zoom help sessions and personal office hours. Zoom is great, | can just set up my 
laptop in front of my whiteboard then it'd be like students sitting here in the office, and they 
could ask questions. | guess basically the big difference is the video's online using Zoom for 
office hours and help sessions. Attendance was about the same. The recitation was required. 
Everybody would come in for the recitation class on Zoom, but for office hours it was just 
as needed as they [students] needed help. It’s about the same as office hours, you now have 
maybe five students a week. 


A mathematics instructor who originally did not plan to hold live virtual office hours described their 
decision to do so. “At first, | thought 
I'd just use the videos that Wiley At first, | thought I'd just use the videos that Wiley 
created,” said the instructor. “Then created, said the instructor. Then my students begged 
my students begged me to hold me to hold WebEx sessions, so | held a WebEx session 
WebEx sessions, so | held a WebEx every day during class time because they still wanted the 


session every day during class interaction. 


time because they still wanted the 
interaction.” When asked what the 
most challenging aspect of remote 
learning was for their students, the instructor reflected, 


| think the ability to ask questions real-time, as they [students] were having the questions. 
Whether it was on homework problems or lecture notes, being able to ask those questions 
immediately, | think that was the hardest for them. That's why they wanted the WebEx 
sessions. 
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Students generally appreciated having the opportunity to work with their instructor online in the ways 
described above. A student taking economics expressed their desire that online office hours continue, “If 
they can make the online or virtual office hours a permanent thing, regardless of whether they go back to 
class in the fall or not, | would really like that because that'd be very helpful.” 


A chemistry student whose course had synchronous recitation (tutorial) sessions explained how those 


sessions helped with the transition to remote learning. 


We met on Zoom and he [professor] said that we would meet every Tuesday like normal for 
recitation to answer questions. He would post the lectures at the start of each unit so there's 
four units, Monday, Wednesday, Friday. We'd watch about 15-minutes’ worth of his lectures 
because that’s like a normal class time. If you do that, you'll be on pace for the test. 


Several instructors observed that some students who had not asked questions in face-to-face class 
sessions started doing so after the course moved online. 


In Zoom, | would ask a question and say, “Okay, everyone who thinks they have an answer, 
drop it into the chat box.” | would see the chat box just start to fill up with answers. Some of 
them right, some of them wrong, they would prompt discussion. Students seemed a lot more 
willing to participate via chat than by speaking up out loud in class. | don't know if because 
they're at home behind a computer screen if that took away some of the embarrassment 
factor or the on-the-spot factor, but a lot of students were much more willing to leave 


comments or answers or responses in a Zoom chat. 


Students seemed a lot more willing to participate Another instructor noticed that some stu- 


via chat than by speaking up out loud in class. dents liked to attend online office hours and 
simply lurk while others asked questions. 


Interestingly, | think a lot of students maybe felt more comfortable with the online office 
hours because they could just log on, mute their volume, not put on the video, and they 
could just listen, and they didn’t have to physically go show their face anywhere or sit with 
their peers. ... Some of them are just shy and they maybe never would have come to Office 
hours in person, but the fact that they can be completely anonymous and just be present and 
listen but not have to actually show their face or talk helps some of them to participate. 


Other instructors tried to be responsive to their students’ questions about course content by inviting 
students to email them with questions, 


| would respond to anyone who emailed me with concerns. | put announcements to every- 
one reminding them [students] and | posted in Canvas each week, a list of what they should 
do each day ... | didn’t tend to email students individually unless they had emailed me first. 
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When you're lecturing [in person], even though they're not talking, you get a sense of what 
they're getting and what they don't get. They'll come up to you after the lecture and talk to 
you. You don't get any of that with the remote learning, especially the way that | did it with 
asynchronous. | told them that | was available and that they could email me and that we 
could web chat. 


Students often sent questions to their instructors through email. Their satisfaction with this method 
depended on the instructor's response time and response predictability. A biology lab student expressed 
frustration about the delayed response time getting answers to questions about lab assignments. 


There wasnt consistent communication all the time with the professor. There was just a lack 
of clarity of, “Okay, | have a question now, but when am | going to get this response?" It could 
be anytime between one or two days and the lab is due today. If | have a question about the 
lab, why even bother asking a question because I’m not going to get a response on time. 


Instructors, on the other hand, often felt worn down by student expectations to be available to answer 
questions 24/7. 


Yes, | would say | found myself working a lot more post-COVID than prior to COVID. There 
were a couple of students that expected responses to emails or to questions in the discus- 
sion forums within 10 minutes of them posting it. It‘d be 5:30 at night and we would try to 
eat dinner and I'd be like, “l got to respond to this.” It was tough because there was no, “This 
is the end of the day work balance, and then okay, | can focus on my family.” That was no 
longer there. 


Classes using courseware appeared to better positioned to provide students with the formative feed- 
back they craved during COVID-19. An economics student, for example, explained how such assess- 
ments helped them reflect on content areas that they learned and those that needed additional work: 


She [the professor] had a little quiz at the end of each chapter to ascertain where you were 
in terms of the content and your understanding. Those were very helpful because | was able 
to say, “| dont really get that topic, so | need to go over that again.” She also opened up a 
section on McGraw Hill that allows you to customize quizzes for yourself based on chapters. 
Those were extremely helpful because when | realized, “Well, this topic I'm not getting,” | just 
go and do a random customization of this chapter on McGraw Hill, and it really helped me a 
lot. 


A community college instructor described going to great lengths to communicate with students to make 
sure they were well and able to cope during COVID-19. 
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| don't think I've ever called a student [before COVID-19]. Because in person it’s like, “It’s your 
responsibility and I'm available.” | really try to build that self-reliance because you're going 

to transfer to a four-year school and they're not going to care at the level that we care. ... 
During this [COVID-19] | called and | emailed, and | never did so much of this. | think it really 
did help. Some people really came back. ... | probably did 300% more of that than | would 
ever do. Literally, every week, “Okay, you still haven't logged into the class. What's going on? 
What do you need? Are you okay? Okay. Now we're calling you. 


This instructor used records of student activity in the courseware to identify students who were not 
engaging with the course content: 


| was paying attention like a hawk to everybody's logins. If you haven't logged in in two 
weeks, if two weeks passed and you hadn't logged in, I'm on the phone, I’m emailing you. 
That's why | said some people got caught in that safety net, which is good. They were like, 
“Oh my God. Things have been crazy” or “I'm just not really used to being at home like this 
and I'm off my own schedule,” or whatever it was. That caught a lot of people [who were 
struggling], but not everybody. 


Insights for the Path Ahead 


We have described the nature of courses as they were conducted remotely in response to COVID-19 as 
well as some of the tradeoffs and concerns faculty dealt with in deciding how to run their courses. While 
many press accounts of remote instruction during COVID-19 decried its quality (Hubler, 2020), propo- 
nents of online learning made the point that what was occurring in spring 2020 was emergency remote 
instruction rather than online learning, as it is generally defined and practiced in the field (Gallagher & 
Palmer, 2020; Hodges et al., 2020). 


Designing and implementing an effective online course—or any course for that matter—can be concep- 
tualized as attending to three essential “Cs:" Cognitive, Community, and Communication. 


¢ The Cognitive dimension of a course is the one most faculty are accustomed to thinking about. 
It includes identification of the key learning outcomes for the course, design of meaningful 
assessments of those outcomes, and strategies for encouraging active (“minds on") learning. 


¢ The Community dimension of a course involves building a learning community in which all 
students feel they belong (“rightful presence”) and are supported. Use of real-world examples 
touching on students’ lives can draw on their prior knowledge and promote a sense of 
togetherness. Instructor responsiveness to student questions and needs and the instructor's 
demonstration of respect for all students help build community as do opportunities to learn with 
fellow students through peer learning, study groups, and team projects. 


Finally, instructors should have a plan for the Communication dimension of their course. This 
encompasses being very clear about course expectations and standards for performance, providing 
information on how and where students can get help with course content and can access other 
kinds of support, and personalized suggestions to individual students for how to further their 
learning. 
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Survey Data on Students’ Experience of Recommended Practices 


The need for a quick shift to remote instruction in spring 2020 meant that instructors new to teach- 
ing online would not have time to learn about and implement best practices in online instruction. 
Descriptions from case study course instructors and students provided above paint a picture of what 
instructors with some experience with digital courseware were able to do under the circumstances. 
Responses from the national student survey provide data on a much larger sample of spring 2020 remote 
college courses and indications of just how much they deviated from best practices in online teaching 
and learning. The national student survey included items asking whether students experienced each of 
eight instructional practices abstracted from the higher education research and practitioner literatures 
on effective teaching and learning online (e.g., American College and University Educators, n.d.; Darby, 
2019; Luskin, 2019; Orlov et al., 2020; Salcido & Cole, 2018; Siemens et al., 2015). Specifically, the eight 
practices included on the national survey to represent the three Cs of quality instruction were: 


« Assignments that ask students to express what they have learned and what they still need to learn 
(cognitive) 


¢ Breaking up class activities into shorter pieces than in an in-person course (cognitive) 

e Frequent quizzes or other assessments (cognitive) 

e Live sessions in which students can ask questions and participate in discussions (Communication) 
e« Meeting in breakout groups during a live class (community) 


e Personal messages to individual students about how they are doing in the course or to make sure 
they can access course materials (communication) 


e Using real-world examples to illustrate course content (Community) 


e« Work on group projects separately from the course meetings (community) 


Figure 2 shows the proportion of survey respondents reporting that their course included each of these 
features when it was taught remotely. Because STEM (science, technology, engineering, and mathemat- 
ics) courses tend to use different teaching approaches than courses in other subject areas (e.g., human- 
ities, social sciences) do, student responses for STEM and non-STEM courses are shown separately. 


A majority of survey respondents describing both STEM and non-STEM courses reported that their 
courses included live sessions during which they could ask questions, frequent quizzes or other assess- 
ments, and use of real-world examples to illustrate course concepts. STEM courses were more likely than 
non-STEM courses to employ frequent quizzes or other assessments. 


More students in non-STEM courses than in STEM courses reported that their instructors sent personal 
messages to them and gave assignments requiring them to reflect on what they had learned and what 
they still needed to learn. 


Other recommended practices—meeting in breakout groups during a live class, breaking up class ac- 
tivities into shorter segments, and group projects outside of class time—were used in less than 40% of 
courses overall. Instructors of courses in non-STEM subjects were more likely than instructors of STEM 
courses to break class activities up into shorter segments for online instruction. 
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Figure 2. Proportion of survey students reporting recommended instructional practices in their STEM and 
non-STEM courses 
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Another item on the Survey of Student Perceptions of Remote Teaching and Learning asked students to 
rate their satisfaction with their course after it went to remote delivery on a 4-point scale (very satisfied, 
somewhat satisfied, somewhat dissatisfied, very dissatisfied). Analysis of differences in course satisfaction 
responses for courses with and without each of the recommended instructional practices revealed that 
those practices most strongly related to students’ course satisfaction were: 


e Personal messages from the instructor on how the student was doing or to make sure they could 
access course materials. Sixty-eight percent of students who received messages of this type were 
satisfied with their course overall, compared to 47% of those who did not. 


e The instructor's use of real-world examples to illustrate course content. Among students whose 
instructor did this in the online course, 67% were satisfied overall, compared to a 42% satisfaction 
level among those whose instructor did not use real-world examples. 


¢ Assignments requiring students to express what they had learned and what they still needed to 
learn. Satisfaction with online courses that included such assignments was 68% compared to 50% 
for courses without this feature. 


Student Course Satisfaction Predicted by Instructor Use of Best Practices 


Another analysis of the survey data examined the influence of including multiple recommended practices 
in the course during COVID-19. The number of the eight recommended practices that students said their 
course included was tabulated for each survey respondent. Figure 3 shows that the number of these 
online instructional practices incorporated in the course predicted students’ satisfaction with their course 
during COVID-19. As seen in the top bar in the figure, 74% of students in courses using 6 or more of the 
recommended instructional practices were either somewhat or very satisfied with their course as it 
was taught during COVID-19. 
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Figure 3. Satisfaction with course during COVID-19 by number of recommended online practices used in 
instruction 
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Among survey respondents, staying motivated to do well in the course emerged as the most widespread 
barrier for students after their courses moved online due to COVID-19. The routines of attending class 

in person had provided a way for students to hold themselves “accountable,” and some found that they 
were not prepared for the level of learner self-direction required in an online format, especially when 
classes were held asynchronously. Disruptions associated with work and family life further exacerbated 
the demotivating effects of losing structure and personal interaction. Additionally, emotional distress 
and uncertainty caused by the pandemic undermined students’ attempts to focus on academic work. 
Students from minoritized groups and those from lower-income households were differentially impacted 
by challenges other than motivation. The survey and case study also call attention to the importance of 
college campuses as physical spaces for student success. Campuses provide students with separation 
from personal and work life and thus a place to pursue their studies with fewer daily distractions. They 
also provide facilities upon which students rely, especially WiFi and access to computing devices and 
laboratories. 


Reduced Opportunities for Active and Collaborative Learning 


Active learning activities calling on students to think deeply about course concepts (Freeman et al., 2014; 
Haak et al., 2011; Orlov et al., 2020; Prince, 2004) and social construction of knowledge (Dabbagh et al., 
2019; Vygostsky, 1978) are two of the most effective instructional practices in the research literature. A 
number of specific techniques, such as peer-assisted learning and project-based learning, combine these 
two concepts to engage students in deeper learning (Lewis & Lewis, 2005). Many instructors lacked the 
tools and preparation time needed to design online course activities involving student collaboration and 
active learning. Only 3 of the 29 case study course instructors we interviewed described creating break- 
out groups in Zoom to allow for small-group discussion, for example. None of the instructors interviewed 
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for course case studies described assigning group projects during COVID-19. Even a case study professor 
who typically used a highly interactive, evidence-based pedagogical strategy (process-oriented guided 
inquiry learning or POGIL) for his general chemistry course did not know how to translate this approach 
into an online environment. Thus, while instructors were justifiably wary of placing too many demands 
on students unaccustomed to online course environments, the fact that so many remote courses lacked 
opportunities for even informal group problem solving during class time may have added to students’ 
feelings of isolation. 


The net result of the COVID-19-driven transition to remote instruction was that students were some- 
what less likely to experience instructional practices designed to elicit active learning, and much less 
likely to have opportunities for social construction of knowledge. Many courses, by necessity, reverted 

to more traditional, didactic methods relying upon video lectures and traditional assessments (e.g., 
multiple-choice quizzes and tests rather than performance assessments or complex problem-solving 
requiring transfer of knowledge). These changes in practice may have exacerbated academic challenges 
for low-income and minoritized students, who have been found to benefit disproportionally from active 
learning pedagogies (Freeman et al., 2014; Prince, 2004). 


Strategies Needed for Promoting Equity and Rigor 


Instructors interviewed for the course case studies expressed empathy for their students and described 
ways in which they had relaxed deadlines and some of their normal course requirements in light of the 
challenges some of their students were facing. Instructors were motivated by the desire to reduce the 
anxiety their students were feeling and to be fair to students who did not have the technology access or 
quiet space and dedicated time for learning that they had had before COVID-19. Most students appre- 
ciated these instructor demonstrations of empathy and flexibility, but a downside to these instructor 
responses emerged as well. Without deadlines, some students let their course work slip and had a hard 
time recovering. Issues and trade-offs emerged for course examinations as well. Concern with cheating 
led some instructors (and entire campuses) to require proctoring software, a practice that some students 
found intrusive and anxiety provoking. Use of tight time windows to discourage students from Googling 
the answers to test items was another practice that raised student anxiety, especially for those with poor 
internet connections or disruptive activities within their homes. Open-book examinations without time 
limits, on the other hand, were sometimes perceived by students as requiring less preparation and raised 
student concerns that their peers were cheating. 


Increasing Consistency Within and Across Courses 


Another theme found in student comments concerned the benefits of consistency (and confusion 
caused by inconsistency) in course expectations before and after the shift to remote instruction. Student 
and instructor interviews suggested that use of adaptive and interactive courseware systems smoothed 
the transition online because this component of courses was unchanged and provided structure. 
Students in courses featuring adaptive courseware were already accustomed to doing coursework online 
and knew how to seek help within the systems (for those that provided intelligent tutoring). In other 
words, the learning curve necessitated by COVID-19 was less steep for both instructors and students 

in courses that were already using online learning systems as part of their courses because the remote 
version of the course was largely consistent with the version taught pre-COVID-19, and students were 
already accustomed to working independently with the courseware. 
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Students would have liked consistency also in the way- 


their different courses were structured. The toll taken 2 ‘ 
Some instructors said they 


by lack of consistency across courses was highlighted : 
eye] (eM al-\Vom-] 0) e)a=relf-}«=10| 


by anumber of students who pointed to the fact that 


the different courses they were taking used different more oleexxela] el di=) guidance 
approaches, routines, and technologies. This occurred ideoleamealcila departments 
because instructors, even within the same department, about how to structure their 


typically designed their courses in isolation, making 


online courses. 


different decisions about how lectures were delivered, 


whether to hold online office hours, and how to 
conduct assessments online. Some instructors said they 
would have appreciated more prescriptive guidance from their departments about how to structure their 
online courses and which technology tools to use. 


There's a lack of coordination among faculty and even though there's e-learning professional 
development, there's just a severe lack of coordination among faculty at [institution] in terms 
of delivery of content for our students. It didn’t surprise me that the administration was like, 
“Just do it.” and everyone just designed their courses in their own way as they saw fit, without 
a lot of guidance and without a lot of pedagogical background into what they're doing. 


Some instructors also pointed out that institutional laissez faire policy on how to handle the transition 
to remote instruction placed a burden on students to employ many different technology tools and keep 
track of instructors’ different course structures and requirements. As one instructor commented, 


| would have appreciated if our department as a whole said—maybe even the whole school— 
would've said: “This is how we're transitioning. You are required, as a faculty member, to 
meet with your students and go through the lecture or you're required to have office hours 
and pre-recorded videos.” Because | feel the students when they were coming back, they 
were just like, “What am | even doing?” They were receiving all these e-mails, and they were 
getting confused between, well, “In this class I'd have to do the lectures and | have to watch 
them beforehand, but in this class, | have to a different task.” | just think the students had a lot 
of confusion as to what the classroom was going to be. 
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Recommendations 


As the COVID-19 pandemic continues and many students continue to be faced with the choice of taking 
courses online or not taking them at all, the spring 2020 experiences of students and instructors in STEM 
courses suggest steps that colleges and instructors should take. 


For Higher Education Institutions 


At the institutional level, as long as there is a possibility that students will have to take their courses 


entirely or partially remotely, we recommend: 


e Ensuring every student has a computer adequate for videoconferencing, the learning management 
system, and any digital learning system used in the college's courses. Some colleges are providing 
technology to students who need it or making the technology a basic requirement so that it can be 
covered by student scholarship and loan money. 


¢ Providing both synchronous and asynchronous learning options within each course to provide 
the flexibility some students need without sacrificing the sense of instructor presence and support 
most students find motivating. 


e Using institutional resources to communicate with students through multiple channels and to 
ensure continuation of support and resources such as tutoring and supplemental instruction if the 
campus has to close. 


e Establishing expectations at the institution or department level about online course structure to 
ensure greater consistency in student experience across courses in the event that instruction must 
be done at a distance. 


e Providing support for faculty members to learn how to incorporate evidence-based pedagogical 
practices into online instruction. In particular faculty need support in developing online analogs 
for lab activities, using tools that enable student collaboration, and designing assessments that 
emphasize the application of course content rather than memorizing factual content that can be 
looked up online. 


¢ For large, introductory courses, staffing the course with a second instructor or teaching assistant 
is desirable in light of the combination of teaching and technology challenges in delivering high- 
quality instruction online. 


At the same time, institutions and departments need to be thinking about how students who took their 
courses during spring 2020 will be served when taking future courses in their course sequence. Student 
reflections on their preparation for future courses and instructor reports of reductions in course cov- 
erage and relaxed grading mechanisms suggest that the spring 2020 COVID-19 learning experiences 
may have long-term consequences for students if additional supports are not offered. We recommend 
that departments work with faculty teaching the courses that students take after their introductory 
course to: 


¢ Develop plans for dealing with broader-than-usual ranges of prior knowledge and competencies 
among students enrolling in the course. Implementing strategies such as supplemental instruction 
and adaptive learning courseware can help students who took their introductory courses during 
the COVID-19 pandemic catch up on what they may have missed or failed to master. 
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¢« Double down on tutoring and supplemental instruction aligned to specific key courses. 


For Instructors and Course Designers 


For course instructors and those who help them design their courses, we offer two overarching 
recommendations. 


¢ First, consider incorporating independent digital learning resources and activities into courses 
even if face-to-face instruction resumes. These activities provide opportunities for self-regulated 
learning that prepare students for future learning and can be adapted to individual student needs. In 
the event that circumstances change and students have to do some of their learning at a distance, 
they will be better prepared to continue their coursework and the transition to online instruction 
will be less disruptive. 


¢ Second, all faculty should be familiar with techniques for fostering cognitive engagement, 
community, and communication online. |t is always possible that a course may have to shift to 
online instruction even if that was not the original intent. We encourage those designing college 
courses to attend to the course's three Cs—Cognitive, Community, and Communication— 
regardless of whether the course will be taught online, face-to-face, or in a blended mode. In the 
Cognitive realm, areas we view as in need of greater attention include (1) techniques for fostering 
active learning online, especially in courses with large enrollments, and (2) meaningful, “Google- 
proof” learning assessments that call for the application of Knowledge in solving problems. In 
the Community realm, we urge course designers to develop a community-building strategy for 
their courses. Especially in times of crisis such as the COVID-19 pandemic, students’ emotional 
connection to their instructor and peers and their feeling of belonging in the course are important 
in motivating them to learn. To the extent the courses in our case studies are representative, 
courses appear to make little use of techniques that have students work together. Peer learning 
in breakout groups and techniques for establishing an online “presence” can enhance students’ 
sense of belonging to a course community even when learning must be done at a distance. 
Finally, regular communication with students should be regarded as an essential teaching function. 
Instructors should develop communications plans that include how they will repeatedly inform 
students about how to access available resources providing help with the course and other 
kinds of student support, provide clear statements and illustrations of criteria for success, and 
specifications for when and how they will personally reach out to students who are struggling or 
not participating. 


The changes in instructional practices that we recommend can be supported by both general-purpose 
and domain-specific digital learning tools, which are increasingly available for subjects such as eco- 
nomics, mathematics, statistics, anatomy and physiology, biology, chemistry, and physics. Simulations 
and virtual reality technology can support some lab activities at a distance. Adaptive courseware offers 
diagnostic assessments of student competencies that can be used to develop individualized learning 
plans for each student, helping faculty deal with the variability in students’ prior preparation. Reflection 
prompts for students to write about aspects of course material they have reviewed but don’t feel they 
understand are built into a number of courseware products. Learning management systems have 
features that support student-instructor and student-student communications. Some learning systems 
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automatically generate lists of students who have not logged in for a specified time period or who have 
fallen significantly behind the expected rate for mastering course objectives. These systems help faculty 
communicate with students by generating template messages that faculty can send as is or edit as they 
choose. We recommend ramping up faculty learning about the capabilities of digital learning tools and 
how best to implement them. 


Whether or not it involves digital learning tools, course redesign and transformational changes in 
instructional practice require a significant investment of time and effort coordinated at the institutional 
or at least the departmental level. The course improvement process will not be fast or easy, but serving 
students equitably and well demands it. 
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Appendix: Survey Methodology 


Ipsos Public Affairs conducted sampling and data collection for the survey via its online, probability-based 
KnowledgePanel®, which administers internet-based surveys to randomly recruited participants in U.S. 
households. Field work was conducted between May 13 and June 1, 2020. Starting May 15, email remind- 
ers were sent to non-responders every two days; those who did not respond by May 29 also received a 
reminder by telephone, and incentives (panel “points” that can be redeemed for prizes in future sweep- 
stakes) were increased, first on May 27 and then again on May 29. 


Of the 1,709 panelists initially responding, 1,014 were qualified undergraduates taking for-credit college 
courses that started with in-person classes and later went fully online in spring 2020. Quality control 
flagged respondents who completed the survey in less than one-third of the median completion time of 
10 minutes, leading to the removal of six cases, for a final sample size of 1,008. Among the 1,008 panelists 
in the final survey sample, 62% reported on courses in STEM subjects. The margin of error for the survey 
is 3.6 percentage points for the full sample. 


Students self-identified in terms of race/ethnicity when they joined the Ipsos survey panel. For survey 
data analysis, response codes to items regarding race and ethnicity were used to form four groups: Black 
Non-Hispanic, Hispanic, White Non-Hispanic, and Multiracial or Other Non-Hispanic. Analyses of survey 
responses compared the experiences of (non-Hispanic) White students to those of Hispanic and Black 
(non-Hispanic) students (referred to in aggregate as “minoritized student groups’) to better understand 
the disparate impacts of the transition to remote instruction. (There were too few students in other race/ 
ethnicity categories to support reliable statistical estimates.) 


Data were weighted via iterative proportional fitting to benchmark distributions of adult undergraduate 
students with less than a bachelor’s degree from the 2018 American Community Survey. Weights were 
trimmed at 0.23% and 99.77% and then rescaled to match the sample size of all respondents. The final 
weights for qualified respondents were extracted from the above weights and re-scaled to sum to the 
sample size of qualified respondents. The average design effect for the final weight is 1.35, for a margin of 
error of 3.6 percentage points for the full sample. Error margins are larger for subgroups. 


Weighted proportions of responses were used to assess Statistical significance via Chi-square tests, where 
p < 0.05 was used as the significance criterion. Data exploration and statistical analysis was completed in 
R Studio version 1.3.959 (R Studio Team, 2020). 
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